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One attribute of experts is that they are likely to learn from their own mistakes. Experts are unlikely to 
make the same mistakes when asked to solve a problem a second time, especially if they have had access to 
a correct solution. Here, we discuss a study spanning several years in which advanced undergraduate physics 
students in a quantum mechanics course were given identical problems in both the midterm exam and final exam. 
Approximately half of the students were given incentives to correct their mistakes in the midterm exam and they 
could get back up to 50% of the points lost on each midterm exam problem. The solutions to the midterm exam 
problems were provided to all students in both groups but those who corrected their mistakes were provided the 
solution after they submitted their corrections to the instructor. The performance on the final exam on the same 
problems suggests that students who were given incentives to correct their mistakes significantly outperformed 
those who were not given an incentive. The incentive to correct the mistakes had the greatest impact on the final 
exam performance of students who performed poorly on the midterm exam. 
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I. INTRODUCTION 

One characteristic of experts is that they are likely to use 
problem solving as an opportunity for learning COS). In par¬ 
ticular, experts are likely to reflect upon their mistakes in their 
problem solution automatically in order to repair, extend and 
organize their knowledge structure. Unfortunately, for many 
students in physics courses, problem solving is a missed learn¬ 
ing opportunity 00. Without guidance, many students do 
not reflect upon the problem solving process after solving 
problems in order to learn from them, nor do they make an 
effort to learn from their mistakes after the graded problems 
are returned to them 00. 

However, closing the “performance gap” between high and 
low achieving students and ensuring that all students excel 
in physics courses are important goals of physics education 
research 11- Prior research suggests that only provid¬ 
ing students worked examples is insufficient □3 and ef¬ 
fective approaches to learning involve engaging students in 
meta-cognition or self-monitoring while they solve problems 
0U El- For example, research suggests that students who 
went through a productive failure cycle, in which they worked 
in groups to solve complex ill-structured math problems with¬ 
out any scaffolding support struggled to learn up until a con¬ 
solidation lecture by the instructor m. However, these stu¬ 
dents from the productive failure condition significantly out¬ 
performed their counterparts from the lecture and practice 
condition on both well- and ill-structured problems on the 
posttests 03 . After the posttest, they also demonstrated 
significantly better performance in using structured-response 
scaffolds to solve problems on a new topic not even covered 
during instruction. Similarly, Schwartz et al. have proposed 
invention tasks to prepare students for future learning 03 - 

It is often implicitly assumed that, unlike students in intro¬ 
ductory physics courses, most students in advanced courses 
have become independent learners. They will invest time to 


learn from their own mistakes, even if the instructors do not 
reward them for fixing their mistakes, e.g., by explicitly ask¬ 
ing them to correct their mistakes by giving them grade in¬ 
centives 00. Contrary to these beliefs, our earlier investi¬ 
gation found that advanced students in a quantum mechanics 
course did not automatically improve their performance from 
midterm to final exam on identical questions even when they 
were provided the correct solutions and their own graded ex¬ 
ams 0. There was a lack of reflection by supposedly mature 
students; many students did not make use of mistakes identi¬ 
fied on a midterm exam as an opportunity to repair and bet¬ 
ter organize their knowledge. In individual interviews with 
some of the students, we also found evidence that even in 
these advanced courses, many students do not automatically 
learn from their mistakes and they often resort to rote learning 
strategies for getting through the course. 

These issues are particularly important considering that the 
diversity in the prior preparation of students at all levels has 
increased. Many students need explicit guidance and support 
not only in the introductory courses, but also in the advanced 
courses. One instructional strategy that may help is to explic¬ 
itly prompt students to learn from their mistakes by rewarding 
them for correcting their mistakes. This activity can also help 
them learn to make use of problem solving as a learning op¬ 
portunity. 

II. GOAL AND METHODOLOGY 

Here we report on an investigation focused on how well 
students in a junior-senior level undergraduate quantum me¬ 
chanics course learn from their mistakes on midterm exam 
problems when provided with time and grade incentives for 
re-submitting written corrected solutions after the graded 
midterm exams were returned to them. Using a comparison 
group that did not self-diagnose mistakes on the midterm ex¬ 
ams, we investigated the effects of this self-diagnosis of mis- 
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takes on subsequent problem solving on some of the same 
problems repeated on the final exam. 

The study took place over four years with the same instruc¬ 
tor and textbook and the course was primarily taught in a tra¬ 
ditional lecture format. Students were assigned weekly home¬ 
work throughout the fifteen-week semester. For each course, 
there were two midterm exams and a final exam (all exams 
and homework were identical across the four years). Both 
midterm exams covered only limited topics while the final 
exam was comprehensive. Students were instructed in all of 
the relevant concepts before the exams, and homework was as¬ 
signed each week from the material covered in that week. The 
first midterm exam took place approximately eight weeks af¬ 
ter the semester started; the second midterm exam took place 
four weeks after the first midterm exam. For this study, two 
problems were selected from each of the midterm exams and 
were repeated verbatim on the final exam along with other 
problems not asked earlier. 

In the first and third year of the study (total 33 students), 
students were not provided any explicit incentives to learn 
from their mistakes on the midterm exam (comparison group). 
However, in the second and fourth years (total 30 students), 
students were given a grade incentive to correct their mis¬ 
takes (incentivized group): they could earn up to 50% of the 
points they had lost for the corrections. Students in the in¬ 
centivized group were directed to work on their own while 
correcting mistakes, but were free to use any resources in¬ 
cluding homework, notes, and books for help. Solutions to 
all of the midterm exam questions were available to students 
on a course website except that the incentivized group was 
provided the solutions after the students had submitted their 
corrected midterm exam solutions. 

Our goal was to explore the extent to which students in each 
group use their mistakes as a learning opportunity and whether 
their performance on the same problems administered a sec¬ 
ond time on the final exam (which we call the posttest) is sig¬ 
nificantly better than the corresponding performance on the 
midterm exams (which we call the pretest). In particular, we 
looked for correlation between the midterm exam score and fi¬ 
nal exam score on the four common problems for each group 
(incentivized and comparison groups). 

All problems were graded according to a rubric developed 
by the researchers JbJ with better than 95% inter-rater relia¬ 
bility for 25% of the students graded separately (see the sum¬ 
mary of the rubric used for problem 1 in Table I). As shown 
in Table I, on the problem, each student received an over¬ 
all score, which is the average of the Invoking and Applying 
scores. Each of these general criteria (invoking and applying) 
scores is in turn the average of the specific criteria scores (see 
Table I), which can take on the values 0, 1, 1/3, 1/2, 2/3, or 
N/A. “N/A” arises in situations when certain criteria are not 
relevant in determining a score. 

III. RESULTS 

The comparison group and incentivized group had nearly 
identical average performance on the pretest as shown in Table 
II (analysis of variance or ANOVA shows p=0.972). However, 
the distribution of posttest performance of the incentivized 


TABLE I. Summary of the rubric used for problem 1. In this prob¬ 
lem, students are asked to find the expectation value of an observable 
Q in terms of eigenstates and eigenvalues of the corresponding oper¬ 
ator. 


General 

Criteria 

Specific Criteria 

Invoking 

appropriate 

concepts 

Spectral decomposition expressing identity operator 
in terms of a complete set of eigenstates \ipn)- 

i = E liM (Vvl 

Or expressing general state in terms of the eigen¬ 
states of Q: 

\ip) = J2 c n li’n) , where c„ = (ip n \ip) 

Make use of Q \ip n ) = A„ \ip n ) 

(l/j-nW* = (lp\lpn) 

Using legitimate principles or concepts that are not 
appropriate in this problem. 

Using invalid principles or concepts (for instance, 
confusing a general state \ip) with an eigenstate 

l^n»- 

Applying 

concepts 

Inserting spectral decomposition into the expression 
for expectation value 

Eigenvalue evaluated and treated as number. 

Probability expressed in terms of and 


TABLE II. Pretest, posttest, and gain for students in the comparison 
and incentivized groups broken down into low, medium and high per¬ 
formance categories based on students pretest score and for all stu¬ 
dents. While the pretest scores are comparable for the comparison 
and incentivized groups, the posttest scores are significantly higher 
for the incentivized group. The gap between low and high categories 
shrinks from 42.2% in the comparison group to 19.9% in the incen¬ 
tivized group. 

Comparison Group Incentivized Group 



(TV = 33) 



o 

co 

II 

Pre 

Post 

Gain 

Pre 

Post 

Gain 

Low 34.6 

50.8 

+16.2 

33.2 

78.2 

+45.0 

Medium 64.7 

66.3 

+ 1.6 

67.3 

88.5 

+21.3 

High 96.0 

93.0 

-2.9 

96.9 

98.1 

+ 1.2 

All 67.9 

71.5 

+3.6 

67.6 

88.4 

+20.8 


group is significantly better than the comparison group with 
a p-value of 0.001. Table II also shows that generally the stu¬ 
dents with lower performance on the pretest benefit more from 
the explicit incentive to correct their mistakes in the pretest. 

Figure 1 shows the average gain (defined as the arithmetic 
difference between posttest and pretest scores; gain can there¬ 
fore range from -100% to +100%) vs pretest score (which can 
range from zero to 100%) for each student on the four ques¬ 
tions repeated from the pretest to posttest in the comparison 
group and incentivized group. 

The regions of possible gain are shaded according to 
posttest score performance categories: green for High posttest 
performance, yellow for Medium posttest performance and 
orange for Low posttest performance. The performance cat¬ 
egories are defined as follows: High, for scores from 85% to 
100%; Medium for scores from 50% to 85%; and Low for 
scores from zero to 50%. The researchers agreed upon a 50% 
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cutoff somewhat arbitrarily for separating Low and Medium 
categories and 85% cutoff was chosen so that roughly one 
third of the students scored in the High performance category 
on the pretest. 

Figure 1 and Table II show that students with poor per¬ 
formance on the midterm exam were likely to benefit from 
self-diagnosis activities in which they submitted the corrected 
midterm solutions for 50% of the points lost on each problem. 
Therefore, the gap between the High and Low performers on 
the midterm exam was reduced for this incentivized group on 
the repeated problems on the final exam. On the other hand, 
for the comparison group, the gap remained, i.e., scores did 
not substantially improve, and remained diverse. 

The data were analyzed by breaking the students into three 
groups based on their pretest performance as shown in both 
Figure 1 and Table II. The initially high-performing students 
from both the comparison and incentivized groups (scoring 
85% and higher on the pretest) generally performed very well 
on the posttest regardless of the intervention (see Figure 1). 
Most of these students who start in the High pretest cate¬ 
gory stay in that category. Students who initially performed 
at a Medium level on the pretest (scoring between 50% and 
85%) in the incentivized group perform better on the posttest 
than the corresponding students who were in the comparison 
group. In the comparison group, students in the Medium per¬ 
formance category on the pretest were as likely to improve 
on the posttest (above the horizontal axis in Figure 1A) as 
they were to deteriorate (below the horizontal axis). In con¬ 
trast, in the incentivized group, almost all of the students in 
the Medium category on the pretest improved on the posttest 
(see Figure IB). 

Furthermore, about half of these students in the incentivized 
group improved as much as possible on the posttest, saturat¬ 
ing the boundary for maximal improvement (see Figure IB). 
Among the initially Low performing students (pretest scores 
less than 50), many students in both comparison and incen¬ 
tivized groups improved on the posttest. However, the degree 
to which these struggling students performed on the posttest 
is highly dependent on whether or not they received a grade 
incentive to improve. The students in the Low category on 
the pretest in the incentivized and comparison groups had an 
average gain of 45.0% and 16.2%, respectively (see Table II). 
In summary, the gains are much larger for the incentivized 
group, bringing the average of the Low category to the level 
of the Medium category, and the Medium category to the High 
category (see Figure 1 and Table II). 

IV. DISCUSSION 

An easy-to-implement intervention substantially reduced 
the performance gap between low- and high- achieving stu¬ 
dents in quantum mechanics. Explicitly providing grade in¬ 
centives to correct mistakes positively affected the perfor¬ 
mance of students with a diverse spectrum of performance in 
the midterm exam on subsequent solving of the same prob¬ 
lems in the final exam. Our research suggests that even stu¬ 
dents in advanced physics courses such as quantum mechanics 
are more motivated to engage with instructional material in a 
meaningful way if they are provided an explicit incentive El- 



Pretest Score (%) 


Pretest Score (%) 


FIG. 1. Average gain (defined as the difference between posttest and 
pretest score) vs pretest score for each student on the four questions 
repeated from the pretest to posttest in the comparison group (A) and 
incentivized group (B). A few students’ data points overlap. Red- 
filled triangles are for each student from the comparison group and 
the blue-filled squares are for each student in the incentivized group 
in which students received an explicit grade incentive to correct their 
own mistakes on the pretest before answering the same questions on 
the posttest. Students whose scores improved are above the hori¬ 
zontal axis; students whose performance deteriorated are below the 
horizontal axis. 


in). Considering the relative ease with which instructors in 
physics courses at all levels can implement the intervention in 
which students are given grade incentives to correct and learn 
from their mistakes, instructors at all levels should consider 
giving students this opportunity to learn. Asking students to 
correct their mistakes in several courses may also help stu¬ 
dents understand the importance of learning from mistakes 
and the role of appropriate struggle in learning. 

Providing students with incentives to learn from their mis¬ 
takes may positively impact learning without a significant ad¬ 
ditional effort on the part of the instructors and without requir¬ 
ing a substantial initial or recurring investment of resources. 
We find that in quantum mechanics, the corrections them¬ 
selves take very little time to grade because students seize the 
opportunity and produce expert-like solutions even when they 
are given only a few days to correct their own mistakes. We 
note that while students submitted both their initial and cor¬ 
rected solutions, explanations of the mistakes they had made 
and how they corrected them was limited since the instructor 
(who graded them) did not explicitly tell them that they would 
be graded on such explanations. 

While these results are encouraging, caution is urged in in¬ 
terpreting improvement. Our findings support the claim that 
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students improve on problems administered a second time 
when expert-like behavior (self-diagnosing and correcting 
their mistakes) is explicitly incentivized. It does not necessar¬ 
ily follow that students have become adept at self-monitoring 
skills from just two such interventions in the two midterm ex¬ 
ams in quantum mechanics. In particular, we compared the 
performance of students in the incentivized and comparison 
groups on four other problems (on other topics) on the same 
final exam for which incentivized group students did not diag¬ 
nose their mistakes. We find that while the incentivized group 
scored somewhat higher than the comparison group, the re¬ 
sults are not statistically significant on student performance 
on those four other problems on the final exam. 

V. SUMMARY AND CONCLUSION 

We find that the performance of students in the group in 
which no incentives were provided shows that while some 
advanced students performed equally well or improved com¬ 
pared to their performance on the midterm exam on the ques¬ 
tions administered a second time, a comparable number of 
students obtained lower scores on the final exam than on the 
midterm exam. The wide distribution of students’ perfor¬ 
mance on problems administered a second time suggests that 
many advanced students do not automatically exploit their 
mistakes as an opportunity for learning, and for repairing, ex¬ 
tending, and organizing their knowledge structure. On the 
other hand, the performance on the final exam on the same 
problems suggests that students who were given incentives to 
correct their mistakes significantly outperformed those who 
were not given an incentive. The incentive to correct the mis¬ 
takes on the midterm exam had the greatest impact on the final 
exam performance on those problems for students who per¬ 
formed poorly on the midterm exam. 

An explicit incentive to correct their mistakes can be an 
effective formative assessment tool m. Offering grade in¬ 
centives to diagnose and correct mistakes can go a long way 
to close the performance gap between struggling and high- 
performing students. If this type of easy-to-implement inter¬ 
vention is implemented routinely in all physics courses, stu¬ 
dents are likely to use their mistakes as a learning opportunity 
and may even develop better self-monitoring skills over time. 


Considering the relative ease with which instructors in physics 
courses at all levels can implement the intervention in which 
students are given grade incentives to correct and learn from 
their mistakes, instructors should consider giving students at 
all levels such an opportunity to learn. 
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A. AN EXAMPLE SOLUTION 

Below, we provide a student response on problem 1 from 
the incentivized group to show how a typical student improved 
from the pretest to posttest. 

1) The eigenvalue equation for an operator Q is given by 
Q\ijji) = A i IV’i), with i = 1...N. Find an expression for 
(ip\Q\ip), where |i/>) is a general state, in terms of (t/jilip). 

(ip\Q\ip) = Pretest 

N 

= 

2 = 1 
N 

2=1 

N 

= 1 i 1 2 * 4 5 6 7 8 9 10 

2=1 

N 

{4>\QW) = ^2 MW*) (V’#) Posttest 

2=1 
N 

2=1 

N 

= ^2 X i(V#»> (i’iW 

2=1 

N 

= A; i W’iiV’} r 
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